This invention relates generally to intermediate nodes of computer networks and, more specifically, to maintaining order among requests issued over an external bus of an intermediate node of a computer network.
A computer network is a geographically distributed collection of interconnected subnetworks for transporting data between nodes, such as computers. A local area network (LAN) is an example of such a subnetwork; a plurality of LANs may be further interconnected by an intermediate network node, such as a router or switch, to extend the effective xe2x80x9csizexe2x80x9d of the computer network and increase the number of communicating nodes. The nodes typically communicate by exchanging discrete frames or packets of data according to predefined protocols. In this context, a protocol consists of a set of rules defining how the nodes interact with each other.
Each node typically comprises a number of basic systems including a processor, a main memory and an input/output (I/O) system. Data is transferred between the main memory, processor and I/O system over a system bus, while data transactions within the I/O system occur over an external bus, such as an I/O bus. Each bus typically consists of either address, data and control lines, with the control lines carrying control signals specifying the direction and type of transfer, or a pair of unidirectional signals for passing bus packets containing address, data and control such as in the case of HyperTransport (HPT) bus. For example, the processor (i.e., a source) may issue a read transaction to request the transfer of data from an addressed location on an I/O device (i.e., a target) coupled to the I/O bus and over the system bus to the processor. The processor then processes the retrieved data in accordance with instructions that may have been obtained from main memory. The processor may thereafter issue a write transaction requesting that the results be stored in, e.g., an addressed location in the main memory.
Some buses operate in an xe2x80x9catomicxe2x80x9d manner such that the source is granted exclusive access (i.e., control) to the bus until the transaction is complete. However, an atomic bus may potentially waste bus cycles, particularly when waiting for data in response to, e.g., a read request. In a split transaction bus, on the other hand, the source relinquishes control over the bus once the request is sent and an independent response to the request is subsequently returned to the source. Here, the target acquires control of the bus to return the response to the source. The split transaction bus thus essentially enables a transaction to be divided into at least two transfers: the request and the response.
Devices coupled to the split transaction bus typically include common sets of resources (such as buffers or queues) used to store request and response transfers sent over the bus. It is possible that some of the resources may be consumed by these transfers, thereby causing a deadlock situation on the bus. To obviate such a situation, transactions sent over the bus may be reordered. However, re-ordering of transactions over a split transaction bus may result in inconsistent data accesses that, in turn, may adversely impact performance of the system and node.
For example, a transaction re-ordering situation that may result in inconsistent data accesses is a read transaction that requests xe2x80x9cpre-fetchingxe2x80x9d of contents of a block of address locations. If a write transaction is allowed to xe2x80x9cpassxe2x80x9d the read request, then the actual read access will retrieve data having a value indicative of the results of the write request. Another situation where it is necessary to maintain the order of read requests to avoid inconsistent data accesses involves requests directed to a similar address, e.g., the xe2x80x9cheadxe2x80x9d of a first-in, first-out (FIFO) queue. The data xe2x80x9cwordxe2x80x9d present at the head of the FIFO changes between an initial read request and a subsequent read request. Accordingly, if those requests are allowed to get xe2x80x9cout of orderxe2x80x9d, the result of the initial read request would be a subsequent data word rather than the intended initial data word.
In the case of a split transaction bus, such as the conventional peripheral computer interconnect bus or HyperTransport (HPT) bus, it is possible for transactions to be reordered due to the use of virtual channels and ordering rules defined for the bus. In particular, it is possible for a read request followed by a write request to the same address to appear xe2x80x9cout of orderxe2x80x9d at their destination. For example, assume a target on the HPT bus includes a control status register (CSR) that contains status information for use by a source on the bus. Assume also that the source stores (via a write request) a particular value in the CSR and then subsequently retrieves (via a read request) the content of that CSR location to determine if it is appropriate to issue another transaction. An example of a typical instruction code sequence for this application is:
Write MACREG1 less than value greater than 
Read MACREG1
If (bit 3=j) then . . .
Write MACREG1 less than new value greater than 
The first write instruction is directed to writing a register (MACREG1) with a particular value. The next instruction is directed to reading the value of that register to check status; e.g., if (bit 3=j) then, etc. Thereafter, a subsequent write instruction is executed to write a new value to the MACREG1 register. Because of the ordering rules on the HPT bus, it is indeterminate as to which operation will occur first. Specifically, the HPT ordering rules allow write requests (e.g., posted requests) to xe2x80x9cpassxe2x80x9d any other request (e.g., non-posted requests) thereby allowing the transactions to be reordered at a device (such as a repeater) on the HPT bus to avoid deadlock. Thus, it is possible for the subsequent write request to pass the pending read request. Clearly, this may be problematic.
A typical solution to this problem is to require completion of the read request before the write request is issued. That is, to ensure ordering between transactions that may be reordered, software (such as a HPT device driver executing on a processor) is configured to defer (xe2x80x9chold offxe2x80x9d) issuing the subsequent write request until the pending read request completes. However, this approach is inefficient and degrades performance of the system since the processor is likely to stall program execution until the read request completes. The present invention is directed to a technique that ensures ordering of transactions without adversely impacting system performance.
The present invention relates to an apparatus and technique for off-loading responsibility for maintaining order among requests directed to a same address on a split transaction bus from a processor to a split transaction bus controller of an intermediate network node or any general-purpose processor subsystem, thereby increasing the performance of the processor. To that end, the present invention comprises an ordering circuit that enables the controller to defer issuing a subsequent (write) request directed to an address on the bus until all previous read requests directed to the same address complete. By off-loading responsibility for maintaining order among requests from the processor to the controller, the invention enhances performance of the processor since the processor may proceed with program execution without having to stall to ensure such ordering. The ordering circuit maintains ordering in an efficient manner that is transparent to the processor.
Specifically, the ordering circuit checks each write request that is issued by the processor to the split transaction bus to determine if there is a read request directed to the same address currently in progress (i.e., outstanding) over that bus. The circuit includes a data structure (i.e., table) configured to track outstanding requests issued over the split transaction bus. If it is determined that the write request is directed to an address of an outstanding read request, a conflict detection circuit asserts a match bit corresponding to that outstanding read request. Notably, a match bit is asserted for each outstanding read request directed to the same address as the write request. Each write request and its associated set of match bits are loaded into a first-in first-out (FIFO) buffer adapted to store commands to be transmitted over the split transaction bus.
When a write request having any of its match bits set is at the xe2x80x9cheadxe2x80x9d of the FIFO, it must wait until all previous read requests directed to the same address as the write request are completed before it can be sent over the bus. In other words, the read request corresponding to each asserted match bit must complete before the write request is sent over the bus. If additional read requests issued by the processor are directed to the same address, they must wait until the write request has been sent to the bus before they can be sent. In this case, the additional read requests may be loaded into the same FIFO as the write request. However, if additional read requests issued by the processor are directed to addresses different from that of the write request, they may be sent to the bus while the write request waits for the xe2x80x9cconflictingxe2x80x9d read requests to complete.
Advantageously, the present invention increases performance of the node by xe2x80x9cholding off xe2x80x9d a current write request only as long as is necessary. That is, rather than holding off the write request in the presence of any pending read requests, including those requests directed to different addresses, the present invention holds off the write request only until completion of those read requests that have the same, conflicting addresses.